1. Field of the Invention
This invention relates to the field of optical feature recognition, and more particularly to a system and method for automatically interpreting and analyzing gauges, readouts, the position and state of user controls, and the exterior of a vehicle, such as an aircraft, including the position and state of flight control surfaces, in an environment with highly dynamic lighting conditions.
2. Description of the Related Art
The recording and automated analysis of image data is well known in the prior art. For example, optical character recognition, or OCR, is the process of analyzing an image of a document and converting the printed text found therein into machine-editable text. OCR programs are readily available and often distributed for free with computer scanners and word editing programs. OCR is a relatively simple task for modern software systems, as documents are typically presented with known lighting conditions (that is, an image of dark text on a light background, captured with the consistent, bright exposure light of a document scanning system) using predetermined character sets (that is, known and readily-available character fonts).
Systems attempting to recognize handwritten text have the added challenge of handling the variations in personal handwriting styles from one person to the next. Still, these systems often require that the writers print the text instead of using cursive and that they follow certain guidelines when creating their printed characters. Even in these systems, where the individual style variations must be accounted for, the lighting conditions used to capture the text images are well-controlled and consistent.
Another example of automated image analysis is facial recognition. A facial recognition system is a computer application for automatically identifying a person from a digital image of the person's face. Facial recognition programs are useful in security scenarios, such as analyzing passengers boarding an aircraft in an attempt to identify known terrorists. A typical facial recognition program works by comparing selected facial features from the image, such as the distance between the person's eyes or the length of the nose, against a facial feature database. As with optical character recognition, facial recognition works best in controlled lighting conditions when the subject matter (that is, the face) is in a known orientation relative to the image.
It is also common to use video cameras in the cockpit of an aircraft or cab of a land-based, marine or other vehicle as a means of gathering data. In the event of an incident, such as a crash or near-miss, the recorded video can be post-processed (that is, processed by experts and systems off-board the vehicle, after the image data has been downloaded to an external system) to determine what conditions were present in the vehicle during the incident. Storing the video data on board the vehicle requires a large amount of storage space. Because of this, mechanisms are often used to limit the amount of storage required on board the vehicle, such as only storing the most recent video data (for example, only storing the most recent 10 minutes of data, and overwriting anything older than this.)
Cameras can also be mounted to the exterior surface of a vehicle to capture images while the vehicle is in motion. Image and video data of the vehicle's exterior surface, including the position and state of the vehicle's control surfaces and lights, can be relayed to a monitor near the operator of the vehicle. This image data can be recorded in the same manner that image data is recorded from the cockpit or cab of the vehicle, as previously described. The external image data thus captured is subject to the same storage and quality limitations inherent in the storage of image data from the interior of the vehicle.
The ambient lighting conditions of both the interior and exterior of a vehicle are highly dynamic, and vary based on the time of day, the angle of the vehicle in relation to the sun, and on the presence of other external sources of illumination. One portion of an instrument panel or vehicle control surface may be concealed in shadow, while another portion is bathed in direct sunlight. The dividing line between dark and light constantly changes as the vehicle maneuvers and changes position in relation to the sun. Commercially available camera systems for use in vehicles do not perform well in these conditions, and provide low-quality images. These limitations make the task of post-processing the image data to clearly identify details within the images difficult if not impossible.
A single clear image of an aircraft cockpit, however, would contain a wealth of information about the ongoing flight. An image of a cockpit would capture a snapshot of the current state of each of the flight instruments, the position of the pilot and copilot, and the presence of any unusual conditions (such as smoke) for any given moment in time.
Similarly, a clear image of the exterior surfaces of an aircraft or vehicle would capture the current state of items such as control surfaces (rudder, elevator, ailerons, flaps, landing gear, etc.), vehicle lights (headlights, turn signals, etc.), and other vehicle components (doors, windows, wings, etc.).
If automatic image analysis of this image data could be consistently performed in real time, while the trip is in progress, this visual information could be interpreted and stored as numeric data and/or communicated to the operator and/or other onboard systems. Further, if this image data could be captured by a self-contained camera module with built-in processing capabilities, the ability to process and analyze interior and exterior image data could be added to any vehicle, regardless if that vehicle had its own onboard computer or sensing systems. This stand-alone camera module could capture the image data while the trip was in progress, analyze the image data and convert it to numeric data, and then compare that numeric data to pre-existing data, such as a flight plan or terrain model, already contained in the camera module.
What is needed in the art is an imaging system which can, in real time, capture high quality images of an aircraft or vehicle or portions thereof, compensate for the dynamic lighting conditions that can be present, analyze the image data and translate it into numeric data, and provide information and/or advisories to the operators and other onboard systems. This system should also incorporate other information and capabilities such that it is aware of its own position and orientation in three-dimensional space and such that it can operate as a stand-alone unit, without the need to be tied into other onboard vehicle systems.